100 



User(s) / 
Machine(s) 



105 



Analog/ 
DigitalI/0 



152 



Wireless 
(IR) I/O 



153^ 



Special/ 
Custom I/O 



101 



114 



Other I/O 
Control 



112 



Recognition 
Engine 




111 



Command 
Creator 



113 



Command 
Converter 



103 



131 



Synth 
Engine 



Other I/O 



n?vV Control 



Cmd/Data 
Interpreter j — 



Store 
^ 



116 



104a 



125 



Network)™- i 



User(s) / 
Machine(s) 



Server-N 



104 




Server-N 



— \ 
Ka-- 



121 



Machine 



125a User(s)/ 
r— Machine(s) 

126 



^ Portion- 1 [ -p»{ Control-N 

A/ . 



122 



'^122a 



Capability-N 



Feedback-N 



122b 



122c 



Portion-N 

"^7 

123 



Control-N 



*^123a 



Capability-N 



Feedback-N 



123b 



Ka 



123c 



Machine-N 3 



\A 



124 



127 



127a 



127b 



Interfacing 




Operation 


Engine 




Engines 



102 



J 



FIG. 1 



NO 




m 
o 



o 

s 

o 

a 




C5 



CD 
N 
>J 

m 
c 

< 
u 

CO 

o 

a 
a 



r 



O 



a> 

O 

^ CO 

c 

< 



— « CN <^ ^ 

^ ^ •'^ 

^ "^f 

^ s s s s 



CQ 



_o 
c 

CO 

d) 



Q. 



0) <u 

s :> 



0) 



00 

o 




p ^ CS| fTl 

0^ On On 

''^^ ^tj" "sf" 



0) 

c 

ai 
o 

0) 

a> 

Q. 
CO 



is: 



i2 

2 

c 
o 
O 



E 

£ 
o 
O 



£ 

CO 
CQ 



c: 
o 



£ CO 

o E 



OJ 



o 



to vo 00 c7n o '-^ 
r-- CO 00 

t:^ ^ T^- ^ ^ 



0} 



o 



o 



^ sssssss 



c 

c 
3 



03 
o 

CO 



tf> 



CO 

a 

CO 

o 



p 



o 

o 

CO 

> 

< 



J*; 
u 

CO 

<D 
<D 
LL 

> 

< 



E 

CO 
(0 

co 
> 

< 



0) 

u 
c 

CO 

c 
UJ 



Oi 
c 

5 



07 



CQ 

E 

o 

w 

X 



o 

CQ 

c: 

8 

CO 



c 
o 
O 



(0 
CO 

t: 

o 

0) 

O 



CO 

c 

'(0 

Q 



CO 

o 
a 

o 
(/> 
<u 
Q 



o 
CO 



0) 
CO 



I. 



(1) 

-C 



o 



/t 



1— 




<1> 




M 


C 




Q.UJ 


CO 


O. >^ 


An 


< CO 

c 


Q. 


< 


Q. 




< 






C3 
O 



O 




CO 

O 



CO 

£ 

e 

o 
U 



Xi 

a 
O 



C 

o 

t: 

o 
cm 



-a 
c 
m 

E 

E 
o 



(0 



O Q 



to 

ON 




ooooooooOoo 



N 

CD 
C 
< 



c 
Ol 

(A 
0) 
> 

u 
a> 

5' 

O 



c 
LU 

X 

0> 

£= 
O 

O 



CO 

c 

in 



CO 

a. 
O 



LU 

c 
o 



c 
*« 
Q 



c 
LU 



u 

it 

'o 

w 



C3) 

c 

at 

c 
o 

Vi 

c 

X 
LU 



c 
lU 

c 
o 

E 

o 
d 

<0 



LU 



c 
LU 

a; 

< 

O 
w 

w 

E 
O 



Oi 
C 

LU 
c 



O) 

c: 
LU 

o 

O 
CO 
(0 

< 

5 



0) 

o 



o 



• # 




4^p jr(^ /^'^^( J ~^"zrzi 








J — - 



^5 I 




Is 



V 




^1 Ocs/> 





7 RtPorTOo/itf l "^ '^■^ 



— - - " " - — 

7c) 







f ' -yi fgA 



'pi 



\ 




7^7 



1 




i 




7/y 




7fC 



/- 



7 ^ 



73f 




r >i 



K 7j 




Fl6. p 




Lr; \ — , ' ■ — 



/ J ^^^txii y 







\ 

I 



Serial 



030. '2. 








cJI 






/ 



/ 



/ 



Oser ^/d 



7 5l^c!,^^. 




I 




\ 



<; * 



- — ^i<A^ 




C^/^ crs*- ^^^^ 




t 

I- 




< 
o 

CO 



0/ i 
0') 

1/1 



0 

O 




4 



\ 



V2- 



1 



-4- 



00 i ^ 



I 

0 



X 
0 

P 



-3 



CJ 



C 



1 i 



t 



r ^ 



o 



lifP P U Q 



' — f 



D 








.1^ 




"7^ 



H 



[A 



J 




--5 




.0 



C7' 



AW 





Send an Email to [ My Secretary ] 



File Edit View Tools Message Help 



Folder 



Send an Email to [the] 



Send an Email to [ Phone # ] 





e 



CQ 
I 

o 
o 

•a 

■< 



O 
O 

0^ 



T3 



o 

c 
o 

u 



CN m 

I I 1 

-t-i ^ 

^ ^ ^ 

BBS 

C C G 

O O O 

o u u 

ti^ [JU 

O U U 



□ □□□ 

LJ 



(N m 

* • 

u ^ u 

0 a> <u ^ CM 
2 12 2 o, c 

p O O S3 S 

[JLi o c 

1 I I Ut u 

u a u a c 










• 






> 




X 




0) 








O 





"V" 





/ — 






> 


rent 






a 






o 







X 




X 



CO 
0) 





S CD 



LL 



. .2 

a: g 

CL 



> 
(D 



C 

o 



X 
0) 



V 



^ CO 

8 8 

<D p 

















1 



CO 






u ^ ■ 



u 



r 



i. 



i 



t , 



/ 




1 



O.Jd^ 4^L^I Of6^.,^ iJ 




If ^ 



/? 




1 




— fi.^ 





7 




Air ft 



/ 







' f 











105 




User(s) / 
Machine(s) 



\5\M 



Analog/ 
Digital I/O 



Wireless 
^^^ ^ (IR)I/0 



153^ 



Special/ 
Custom I/O 



101 




114 



Other I/O 
Control 



117 



112 



Recognition 
Engine 




111 



Command 
Creator 



113 



Command 
Converter 



115 



Cmd/Data 
Interpreter 



Store 



T7\ 



116 



103 



131 



Synth 
Engine 



132 



Other I/O 
Control 



104a 



104 



125a 




I Server-N 







User(s) / 
Machine(s) 



125a 



Server-N — i 



f Network j 



126 



User(s) / 
Machine(s) 



I Machine —{__ 




Portion- 1 

TTT 

122 



> Control-N 



Capability-N — 



122a 



Feedback-N 



122b 



122c 



Portion-N 

7J 

123 



Control-N 



Capability-N 



123a 



Feedback-N 



123b 



123c 



124 



Machine-N 3 



127 



Interfacing 




Operation 


Engine 




Engines 



127a 



127b 



102 



FIG. 1 



ri 

O 



T3 

o 



a 

>< 

c 
o 

U 



^ sssssssss 



o 
o 

u 



i = 



o 

X 
(L) 

in 



y 

CO 

■4— > 

■»-> 
C/3 



4) 

c 



c 
.2 

■4— > 

5 



eft 




iism 


o 










loq 


S 

<u 


o 


on 


U 





■4— > 

O 

c 
cd 



"is 
'a 



o 



CD 



u 

N 

< 



o 



in in 



^ s s s s 



o 



N 



C 

O 



N 

c 

o 

O 

< 



o 



c: 



{/3 

Cd CC 



• nan C^S 



o 



1-^ CN 



CQ 

CL 

3 

O 
u 

o 

eft 



o 

a 



</3 



^2L 



. ^ cd 



S S S S 



o <l> 

5 g 

Cft > 



0^ $2 

cd a> 
:p S 



O 00 

,cd Cd 

JO 



3 

O 

cft 



)rp 
ties 






User/G( 
operabili 


Securii 
operabili 


Othei 




o 
o 



(N 

o\ 0\ 

^ ^ ^ ^ 



s ss 



00 

o 



O 



O 



c 
• t— < 

c 

D 



Cft 

c 
o 

U 



C (L> 
C 

c Cd 
o b 

U Cd 



c 
o 

o P 
c 



o 



in 00 ON o ^ 

00 00 

^ ^ ^ 

s ssssss 



^ 9. 

cd .in 
9- > 



cft 
O 

o 

a 

cd 
> 



ack 


i 






X) 






king 


eed 


Par 


anc 






x: 


Lin 




> 


En 


> 


< 






< 









o 



1— ( CN CO 

i> 

^ ^ ^ 



in 




I/O Analyzer 



Environment Eng 



Objectives Eng 



Context Eng 



Task/Goal Eng 



Operability Eng 



Designation/Mod Eng 



Specificity Eng 



Extension Eng 



Enhancement Eng 



Cmd Structure Eng 



Linking Eng 



User Assoc Eng 
Other 



J~ 



J~ 
J~ 

s 



s 



402a 

402b 

402c 

402d 

402e 

402f 

402g 

402h 

402j 

402k 

4021 

402m 

402n 



F/6. 46 



I 



111 



(Non-Distrib) 
Command 
Creator 



495 



Command 
Distributer 



I/O 
Portions 



> Commands 



^ Operability 
Portions 



F/G. 4c 



<: iy7Q- tfj 




8^ 







o 




1 — 1 

1 — 


o 


1 — 1 

is: 


1 — 






o o 




o 


1 1 1 










o 


I — 


LU 
LlJ 




Q_ 









<5 



o 

X 




FIG. 7a 

INPUT ENGINE 
511 



FIG. 7b 

COMMAND ENGINE 



512 



£-611 



601 

V 

INPUT CONTROL E 




602 
J. 



SPEECH 
CONTENT 
ANALYZER 



603 



NON-SPEECH 
CONTENT 
ANALYZER 



514 

\ 



FIG. 7c 

APPLICATION SUPPORT 



A S CONTROL \ ^622 



APP/MACHINE 
INTERFACER 



623 

JL 



SPECIAL CMD 
ENG 




CONTROL MUTE/ 
SUPPRESSER 



ROOT-ENG 



612 



CONCAT 



■/linkerP 



613 



r6U 



BASIC OPERATION 
ENGINE 



OPERATION 
ENHANCEMENT 
ENG 



615 

u 



(-616 



DATA ENHANCE. 
ENGINE 



(-617 



FEEDBACK ENG. 



r6l8 



ENVIRONMENT ENG. 



FIG. 7d 



DESIGNATION ENGINE 
515 

\ 631 

i— 

DESIGNATION CONTROL 



- EXPLICIT DESIGNATION 



IMPL DESIG. RESOLVER 
(E.G. BASE, TASK, CONTEXT 
POSITION, ETC.) 



("IMPLIED DESIGNATION 
-EXPLICIT/ALIAS 
INCLUSIVE RELATIVE 
-ANTI -ALIAS 

-SUBJECT, OBJECT, USES 
•-CUED 



632 



-.633 

P 



DATA 



634 



P 



635 



P 



1 636 

cm INPUT RESOLVER \j 



516 



FIG. 7e 

ENHANCEMENT ENGINE 



\ 

I r641 



r 



E.G.< 



P 
^643 



V 



MACHINE SWITCHER 



644 



QUERY ENG \^ 
^645 



DATA CARRIER/ 
FILLER 



646 
U 




MACHINE CTRLS 



P 




521a 



\ 



8a 



701 

USER ID 

USER IDENTIFICATION U^'™/ 



''^TIG. 8b 



711 



CONTROLLER ID 



VOICEPRINT/ 
BIOMETRIC ID 



CONTROL ID 



QUERY USER 



STATUS ID 



LOCATION/ 
MACHINE ID 



FIG. 8c 



\ 



SYSTEM SECURITY 
ANALYZER 



APPLICATION/TASK 
(SECURITY) ANALYZER 



MACHINE OPERATION 
ANALYZER/FILTER 



INFO RECOGNITION 
ANALYZER FILTER 



INFO RETENTION 
ANALYZER FILTER 



INFO DISBURSE. 
ANALYZER FILTER 



ENCRYPTION- 
CERTIFICATION ENGINE 



702 



703 



\^704 
Y705 
\^706 



707 




708 





731- 
732- 



REGISTER 



7JJ- 

CONTINUE 

722 

734 

■ BIOMETRICS 
723 

■ ID 
724 

CONTROLS 
725 




726 



727 



737- 



APPLICATION 
ENGINE 



Y 



712 



713 



CAPABILITIES 
ENGINE 



714 



EXTENDED 
CAPABILITIES 
ENGINE 



715 



LOCATION 
ENGINE 



716 



FIG. 8d 



RELIABILITY 
ANALYZER 



PURPOSE/USE 
ANALYZER 



COLLOQUIALISM 
ENGINE 



INFLECTION 
ENGINE 



CONVERSANCE RELIAB. 
ANALYZE 



APP-LOCATION- 
PROGRESS ANALYZER 



USER(S) PREFERENCE 
ANALYZER 



524 



FIG. 8e 



CONVERSANCE ANALYZER [ ;:^ 



741 



CONTEXT ENG. "p ^ 



742 




743 



RYTHM. engine"] -^ 



744 



CONTINUITY ENG~| ^ 



745 



USE-INTERACTION 
ANALYZER 



746 



USER 
GROUP 
ROLE, 
USE, S 
ETC. 



FIG. 8g 



527 



RETRIEVER ENG. \ ^ 



761 



LOCATOR \ ^ 



762 





FIG. 8f 



525 




USER PREF-SEHINGS ENG 



751 



CONVERSANT PREF ENGINe| -^ 



752 



tendencies m^ ^^-^ 



PROGRESS ENG. 



754 



CONCATINATOR/SPLITTERf -^ 



755 



PREDICTOR RELIABILITY [ -^ 



756 



765 



FIG. 6a 





LOCAL 
SYSTEM 




REMOTE 
SYSTEM 



















FIG. 6b 




COMMANDS 
(DICTATION) 



OTHER 
I/O 
DEVICES 



SYNTHETIC/ 
RECORDED 
SPEECH 

INPUT 



OUTPUT 



- VOICE 




SPEECH 


INTERFACE 




PROGRAM(S) 



NON-VOICE 
INTERFACE 



I/O 
INTERFACE 



UNDERLYING 
APP(S) 



DIGITIZER 



D/A 
CONVERTER 



RECORDER 




SYNTHESIS 
ENGINE 



COMMAND 
INTERPRETER 



UNDERLYING 
APPLICATION 



FIG. 6c 



VOICE 
INTERFACE 



VOICE 
COMMAND 

OPERATIONAL 
CODE 



L 



ANALOG AND/OR 
DIGITAL DATA 



FIG. 6d 



REMOTE 




SYSTEM 






APP 









LOCAL 
SYSTEM 



SPEECH 
ENGINE 






PARAMETERS 






(USER) 




REMOTE 


SYSTEM 



FIG. 6e 




SINGLE USER 
MACHINE 



V.I. 



r 




q5 






00 




CH 


oo 

LU 


NE 


33d: 




o 








AS 


LU 



^oo 



o8 



I — I oo 

I — 



o ^ 

Q O 



O LU 
LU O 
LU <3: 
Li_ 

00 
I 




OO 

LU O ^ 



LU 

oo 



UJ 

00 



o 












o 








LU 




Q:: 
1 . 1 


UJ 


:ONF( 


ACHI 






O 




LU 
O 




1— 1 

> 





1 




en 




LU 
> 

LU 




00 









00 


LU 






00 


O 




n 


o 


O 

LU 


c3 


ct: 

LU 






00 


o 






n 




USER-2 
MACHINE 


SPEECH 
INFO 








TEXT TO COMMAND 
RECOGNITION ENGINE 



TEXT TO DATA 
RECOGNITION ENGINE 



COMMAND EXECUTION ENGINE 



DATA INPUT ENGINE 



FIG. 8 



FIG. 5 

(VI CONVERTER) 



CONVERSATIONAL 
COMMANDS - 



TASK A- 


-1 - 


• 
• 
• 


TASK A- 


■N ■ 




TASK B- 


-1 


• 
• 
• 


TASK B- 


-N 



INTER. CONTEXT 
TASKS 



CONTEXT A 



CONTEXT B 



VOICE 
INTERFACE 
CONVERSION 
ENGINE 



TO 
VOICE 



COMMAND 
INTERFACE 



FIG. 10 



File 



Edit Views Tools Message 



New 



Reply 



Forward 



Inbox: 



Folder 



□ Mailbox 
■ Inbox 

□ Sent 

□ Work 

□ Jokes 

□ Trash 



Print 



Delete 



Send 



Addresses 



m 



1 



V 



Flog 



To: 



From: 



Subject: 



Tonn 
Tom 
Tom 
Tom 



Bob 
Dave 
Fred 
Brendo 



Gift 
Patent 
Car 
Dinner 



FIG. 11b 



SELECTING MULTIPLE ITEMS 



ENTERING DATA 



TRAVERSING MESSAGE CONTEXT 



FILTERING CONTENT 



SELECTING PROPER COMMANDS 



FIG. 11 A 



USING GENERIC ROOTS 



MANIPULATING THE INTERFACE 



TRAVERSING THE INTERFACE 



SORTING MESSAGES 



MERGING MESSAGES 



INTERACTING WITH THE 
ADDRESS BOOK 



CONTROLLING DATA REMOTELY 



USING ALTERNATIVES 



USING ENHANCEMENTS 



USING SUBSTITUTING 




EMAIL 
EXAMPLE 
COMMAND 
EXECUTION 
MODULES 



FIG. 12 



Re: 

File Edit View Tools Message Help 



To 
Cc 
Bcc [ 



Attach [ 



Subject 




" Send an Email [to] " 













To 
Cc 
Bcc 


My Secretary;^^ ^ 









" Send an Email to [ My Secretary ] " 



Inbox - Outlook Express 



File Edit View Tools Message Help 



Folder 



Inbox 



n. 



Xx@:xx.com l/l/OO Hi. 



HE 



Send an Email to [the] 



" Send an Email to [ Phone # ] " 



Select Recipients 



I IThid 



Cent Folderl 
Cont Folder2 



Contact-n+l 

Contact-n+2 

Contact- n+3 
I I I 



I Bcc 



I Ncont I I Prop | 



BI 



in 



I OK I I Cancel 



In 



Re: 



File 



To 
Cc 

Bed 
Att 



Find People 



c 



Name 
Email 



c 



c 



Address [ 
Phone 
Other 



H I Web Site 



3 



] 



Find Now 



Stop 



Clear All 



] 



1 




Contact- 1 




Contact-2 




Contact-3 








-U 



I Close 



I Bcc 



To 



Cc 



BI 



Bcc 




OK I I Cancel 



t—t 
C 
cd 

I 

o 
CQ 

{/! 

<u 

< 



'o 
o 



O 

c 
o 

U 
U 



m Ti- 

I I I 

■*->-•-•■♦-• 

o o o 

cJ 

c c c 
o o o 

U U U 
Ejm 

U U U 



□ □□□ 
I I 



I 

u 



<N r*-i Tj- 

D (L) o ^ CN 
*0 'O " 



o o o 

{jui IX 



O C 



CJ u u o c 



t/1 
(1> 



o 

C 

o 
U 



0) 
O 

m 



T3 



o 
o 

l-J 



o 
2 

c: 



C4 

o 



o 



I 



4 



y 



< 

4) W 



n 



ej cd ( 



00 





HOST 

ANALYZER 
(INTERFACE 
MAPPER) 



CONVERT 
CONTROL 



INTERFACE GLUE 



APPROACH 
ANALYZER 




CONVERSATIONAL 
FACTOR-CHECKER 











INTERFACE 



KNOWLEDGE BASE 



HOST 
OPERATION 
ANALYZER 

MAP 



UNDER LYING APP 
CHECK/MAP 



OPTIMIZER 
(COMPILER) 



CONTROL 



FIG. 19 

NEW COMMAND/CONVERSION 



FIG. 20 



RECEIVE CMD 



DETERMINE 
DATA AND Dl 


ANY TASK. 
ISIGNATIONS 






DETERMINE ANY PRE, 
CONCURRENT AND/OR 
POST CMD EXTENSIONS 






DETERMINE 
DATA,DESIG 
RELATED 


ANY TASK. 
NATION OR 
FEEDBACK 



EFFECTUATE MACHINE 

OPERATIONS 
CORRESPONDING TO 
TASK, DATA. DESIGNATIONS 

AND EXTENSIONS 



EFFECTUATE ANY 
ADDITIONAL FEEDBACK 



COMBINE/FETCH AS NEEDED 



SUITABILITY PER CONTEXT/DESIGNATION 
eg. SWITCH WINDOW PORTION, WINDOWS, 
MACHINE FOR INTERMEDIATES 



COMBINE/FETCH AS NEEDED 

eg. CONTEXT, PARTIAL CODE. ETC. 



(SUITABLY PER CONTEXT) 
r- PRESENT NEEDED WINDOWS IN 
NEEDED FORM 

- PRE COMMANDS (eg.DOWNLOAD/INSERT,0UTPUT) 

- COMMANDS & FUNCTIONS 
SELECT CORRESPONDING DATA 
& FUNCTIONS 

^ POST CMD 

e.g REPOSITION GUIDE 

FEEDBACK 

eg. HIGHLIGHT SELECTION{S) 




END 



Continue/ repose/ rvthem 



FIG. 21 




dispose of finally 

dispose of tennporarily (finish later) 



r 



jj- Break nnonnentunn 
Note Display 




L I mental or 
^ f actual 

articulation 



word-1 word-2 
word word 
^et-1 set-2 

new object or just phrase 
new object or just phrase 



Remote Handlino (predominant circumstances) 





List 1 List N 

Multiple list 
multiple highlight 



Divided Form 

or not-highlighted list 





Remote 



Multiple windows 



-Each potentially including data carrying 
-The farms (primarily) better backed up 



Illustrated using 
folders, but 
applies to any 
such re-direction 



Target 




omplex 
(here,2 elements) 



eg: (GOTO <foldername> Messages 



